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A commentary on 

Encoding of aversion by dopamine and 
the nucleus accumbens 

by McCutcheon, J. E., Ebner, S. R., 
Loriaux, A. L., and Roitman, M. F. 
(2012). Front. Neurosci. 6:137. doi: 
10.3389/fnins.2012.00137 

Acute stress influences neural circuits of 
reward processing 

by Porcelli, A. J., Lewis, A. H., and Delgado, 
M. R. (2012). Front. Neurosci. 6:157. doi: 
10.3389/fnins.2012.00157 

The ability to distinguish good from bad 
options, to approach the former and avoid 
the latter, forms the basis of success- 
ful behavior. This ability is expressed in 
value-based decisions, which in turn is 
thought to depend largely on the pro- 
cess of reinforcement learning. In order to 
adaptively determine the value of differ- 
ent actions, organisms need to take exter- 
nal as well as internal states into account 
(e.g., Rangel et al., 2008). For example, 
finding shelter may be more valuable in 
a cold environment than in a warm envi- 
ronment. Internal states can also affect 
valuation, as illustrated for instance by 
salt appetite (Berridge et al., 1984; Tindell 
et al, 2009; Robinson and Berridge, 2013): 
In the normal (non-salt-deficient) state, 
rats do not usually ingest extremely salty 
solutions or approach cues that predict 
them. However, in a salt-deficient state, 
they do. This pattern of behavior is com- 
patible with the notion that state infor- 
mation can have such a profound impact 



on value computation that a previously 
bad option becomes good (Dayan and 
Berridge, 2014). 

Since its inception in the seventeenth 
century, economic choice theory has grad- 
ually come to recognize the importance 
of internal and external states on valua- 
tion. While it was initially thought that a 
given monetary unit was worth the same 
no matter how wealthy one is (Pascal), it 
was later proposed that the value of a given 
monetary unit is greater when one is in 
a state of poverty as compared to one of 
affluence (Bernoulli). In the last century, 
researchers found that our expectations 
also affect valuation and accounted for this 
finding by incorporating reference points 
into value functions (e.g., prospect the- 
ory: Kahneman and Tversky, 1979; see also 
Koszegi and Rabin, 2006). For example, we 
value a salary raise of USD 200 less when 
we originally expected to receive a raise of 
USD 400 than when we did not expect to 
receive any raise at all. Thus, the value of an 
option can also depend on cognitive states. 

The states and other variables that 
influence valuation are manifold and 
include not only financial status and 
expectations, but also mood, emotion, 
motivation, previous learning, and social 
aspects. Still, little is known about how 
state information influences valuation at 
the neural level. Two recent publica- 
tions (McCutcheon et al., 2012; Porcelli 
et al., 2012) addressed this question. 
The two studies used different techniques 
(dopamine voltammetry vs. functional 
magnetic resonance imaging), different 
model organisms (rats vs. humans), and 



different value-impacting state parameters 
(previous learning vs. stress). Despite these 
differences, both studies found that striatal 
value signals are state-dependent. 

In the first study, McCutcheon et al. 
(2012) measured dopamine release in 
the nucleus accumbens shell following 
intra-oral infusion of sucrose. In half 
of the rats, sucrose was rendered aver- 
sive by pairing it with induced nau- 
sea (via injection of lithium chloride 
just after sucrose consumption). In the 
other half of the rats, nausea induc- 
tion and sucrose consumption occurred 
on different days, so sucrose remained 
appetitive. As expected, aversive sucrose 
elicited fewer appetitive and more nega- 
tive orofacial responses. Neurobiologically, 
unlike appetitive sucrose, aversive sucrose 
reduced accumbens dopamine concentra- 
tion compared to baseline, even though 
the sensory properties of the sucrose were 
held constant in the two conditions. This 
finding converges with several previous 
reports of reduced dopamine firing and 
concentrations induced by aversive stimuli 
(for review, see McCutcheon et al., 2012). 
Conversely, appetitive sucrose elicited a 
(weak) increase in dopamine, in line 
with previous voltammetry data (Roitman 
et al., 2008) and a wealth of previous find- 
ings implicating dopamine in reward pro- 
cessing (for a review, see Daw and Tobler, 
2013). 

The reduction in accumbens dopamine 
concentration in response to aversive 
sucrose shows that learning can change the 
value of a primary appetitive stimulus and 
provides a pharmacological foundation 
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to the reduction in striatal activation 
observed in imaging studies in which par- 
ticipants' reward expectations were dis- 
appointed (e.g., McClure et al., 2003; 
O'Doherty et al., 2003; Pessiglione et al, 
2006; Burke et al., 2010; Kahnt et al, 2012). 

In the second study, Porcelli and 
colleagues investigated the flexibility 
of reward processing in response to 
induced stress (an internal state factor). 
They adapted the cold pressor task, an 
established stress induction procedure 
(Schwabe et al., 2008), for use in the 
scanner. They used MRI-compatible gel- 
pacs to subject half of the participants 
to stress-inducing cold temperatures and 
half to room temperature (for details on 
the procedure, see Porcelli, 2014). Both 
groups then performed a card-guessing 
task in which they could win $5 or $0.50, 
or lose $0.25 or $2.50. The asymmetry 
between the gain and loss domains aimed 
to ensure that the impact of reward and 
punishment on behavior and brain activ- 
ity was similar and thus to compensate 
for the fact that people generally exhibit 
loss aversion (another feature of prospect 
theory; Kahneman and Tversky, 1979). 
Throughout the experiment, the level 
of salivary Cortisol (a stress-inducing 
hormone) was measured at 15 minute 
intervals using an oral swab. 

The adapted cold pressor task resulted 
in greater feelings of discomfort and a 
higher Cortisol level in the stress induction 
group than in the control group. However, 
the most striking results from this study 
pertain to the striatal BOLD response: 
Responses to rewards as compared to pun- 
ishments were much larger in the caudate 
and putamen in the control group than 
in the stress induction group. Participants 
in the latter group only showed reward- 
related activation in this region when out- 
comes were of high magnitude, suggesting 
that stress causes a desensitization of the 
reward network. In addition, the reduc- 
tion in response to rewards under stress 
was observed in the dorsal, but not the 
ventral striatum. These regions have been 
shown to correlate with reward learning in 
a manner consistent with computational 
models of reinforcement learning (specif- 
ically, the actor-critic model proposed by 
Barto, 1995), with dorsal actor regions 
processing action contingencies that guide 
future choices, and ventral critic regions 



making predictions about future rewards 
(O'Doherty et al., 2004). If stress causes 
a desensitization to reward in the "actor" 
areas of the striatum, action-outcome con- 
tingencies may not be processed accurately 
enough to guide future choices, forcing 
the organism to rely on more habitual 
responses. This may manifest itself in a 
return to otherwise suboptimal reward- 
seeking behavior, such as the relapses com- 
monly seen in recovering addicts (Everitt 
and Robbins, 2005). 

Moreover, one could hypothesize that 
stress, which by itself increases tonic lev- 
els of dopamine (e.g., Inoue et al., 1994), 
may prevent the detection of phasic reduc- 
tions in dopamine that could be elicited 
by stimuli that predict negative drug- 
related effects (see also Weiss et al., 2001; 
Schultz, 2011). This notion would pre- 
dict that stress reduces sensitivity to losses 
through a dopamine-dependent mech- 
anism. Incidentally, one possible target 
region for implementing this mechanism 
is suggested by the Porcelli study (Porcelli 
et al, 2012), which reports reduced mag- 
nitude discrimination under stress in the 
inferior frontal gyrus. Given that the infe- 
rior frontal gyrus appears to play a role in 
response inhibition and cognitive control 
(e.g., Bari and Robbins, 2013), it might be 
worth investigating the role of dopamine 
on punishment sensitivity under 
withdrawal-induced stress in that region. 

Taken together, the two papers dis- 
cussed here support the notion that reward 
processing in the brain is flexible and 
highly dependent on internal and external 
states. The interaction between the envi- 
ronment and internal states allows organ- 
isms to prioritize their goals and adapt 
their behavior accordingly. We suggest that 
this inherent flexibility can be detrimental 
when reward and learning systems are arti- 
ficially challenged, for example, through 
the use of addictive drugs. While the use of 
drugs themselves may affect internal states 
(such as by tonically enhancing dopamine 
levels), subsequent withdrawal may induce 
stress, causing a change in behavior, e.g., 
in the form of a shift toward habitual 
responding and enhanced drug-seeking. 
Moreover, state processes could interact 
with drug effects and further research 
may wish to investigate how this interac- 
tion contributes to addiction and relapse. 
While the two papers (McCutcheon et al., 



2012; Porcelli et al., 2012) give some leads 
for that endeavor they more generally 
highlight the importance of external and 
internal states for brain and behavior. 
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